Great-deluge hyper-heuristic for examination timetabling

Hyper-heuristics can be identified as methodologies that search the space generated by a finite set of low level heuristics for solving search problems. An iterative hyper-heuristic framework can be thought of as requiring a single candidate solution and multiple perturbation low level heuristics. An initially generated complete solution goes through two successive processes (heuristic selection and move acceptance) until a set of termination criteria is satisfied. A motivating goal of hyper-heuristic research is to create automated techniques that are applicable to a wide range of problems with different characteristics. Some previous studies show that different combinations of heuristic selection and move acceptance as hyper-heuristic components might yield different performances. This study investigates whether learning heuristic selection can improve the performance of a great deluge based hyper-heuristic using an examination timetabling problem as a case study. DOI: 10.4018/jamc.2010102603 40 International Journal of Applied Metaheuristic Computing, 1(1), 39-59, January-March 2010 Copyright © 2010, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. directly producing a solution for whichever search problem we are studying. Several hyper-heuristics approaches have been proposed in the literature. It is possible to consider methodologies based on perturbation low-level heuristics and those based on construction low-level heuristics. The latter type builds a solution incrementally, starting with a blank solution and using construction heuristics to gradually build a complete solution. They have been successfully investigated for several combinatorial optimisation problems such as: bin-packing (Tereshima-Marin et al., 2007), timetabling (Terashima-Marin et al., 1999; Burke et al., 2007, Qu et al., 2008), production scheduling (Vazquez-Rodriguez et al., 2007), and cutting stock (Terashima-Marin et al., 2005). On the other hand, approaches based on perturbation heuristics find a reasonable initial solution by some straightforward means (either randomly or using a simple construction heuristic) and then use heuristics, such as shift and swap to perturb solution components with the aim of finding improved solutions. In other words, they start from a complete solution and then search or select among a set of neighbourhoods for better solutions. A class of the most commonly used hyper-heuristics based on perturbation (improvement) low level heuristics is the choice hyper-heuristics (heuristic selection methodologies). They have been applied to real world problems, such as, personnel scheduling (Cowling et al., 2001; Burke et al., 2003b), timetabling (Burke et al., 2003b; Dowsland et al., 2007), and vehicle routing problems (Pisinger et al., 2007). In a choice hyper-heuristic framework based on perturbation low level heuristics, search is mostly performed using a single candidate solution. Such hyper-heuristics, iteratively, attempt to improve a given solution throughout two consecutive phases: heuristic selection and move acceptance as illustrated in Figure 1. In Figure 1, a candidate solution (St) at a given time (t) is modified into a new solution (or solutions) using a chosen heuristic (or heuristics). Then, a move acceptance method is employed to decide whether to accept or reject a resultant solution (Rt). This process is repeated until a predefined stopping condition is met. Only problem independent information flow is allowed between the problem domain and hyper-heuristic layers. Unless, we specifically say otherwise, a choice hyper-heuristic refers to a hyper-heuristic that operates on a set of perturbation low level heuristics from this point onwards. Moreover, such a hyper-heuristic will be denoted as heuristic selection − move acceptance based on its components. Great deluge is a well-known acceptance strategy (Dueck, 1993; Burke et al., 2003). Bilgin et al. (2007) reported that hyper-heuristics formed by different combinations of heuristic selection and move acceptance methods might yield different performances. Moreover, simple random−great deluge delivered a similar performance to the best approach; namely, choice function – simulated annealing for examination timetabling. Obviously, simple random receives no feedback at all during the search to improve upon the heuristic selection process. Hence, in this study, great-deluge is preferred as the move acceptance component within a choice hyperheuristic framework to investigate the effect of learning heuristic selection on its overall performance for solving the same examination timetabling problem as formulated in Bilgin et al. (2007). The learning mechanisms, inspired by the work by Nareyek (2003), are based on weight adaptation. 2 hyper-heuristics and learning Although hyper-heuristic as a term has been introduced recently (Denzinger et al., 1997), the origins of the idea date back to the early 1960s (Fisher et al., 1961). A hyper-heuristic operates at a high level by managing or generating low level heuristics which operate on the problem domain. Meta-heuristics have been commonly used as hyper-heuristics. A hyper-heuristic can conduct a single point or multi-point search. Population based meta-heuristics which perform multi-point search, such as learning classifier systems (Marín-Blázquez and Schulenburg, 2005), evolutionary algorithms (Cowling et al., International Journal of Applied Metaheuristic Computing, 1(1), 39-59, January-March 2010 41 Copyright © 2010, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. 2002; Han et al., 2003; Pillay and Banzhaf, 2008), genetic programming (Keller et al., 2007; Burke et al., 2009a), ant colony optimisation (Cuesta-Canada et al., 2005; Chen et al., 2007) have been applied to a variety of combinatorial optimisation problems as hyper-heuristics. Distributed computing methods can also be used to perform multi-point search (Rattadilok et al., 2004; Rattadilok et al., 2005; Ouelhadj et al., 2008). Özcan et al. (2008) presented different hyper-heuristic frameworks showing that a matching performance to memetic algorithms can be achieved. In this study, the choice hyper-heuristic framework as presented in Figure 1 is studied. The primary components of such hyper-heuristics are heuristic selection and move acceptance. A major motivating feature of hyper-heuristic research is the aim to facilitate applicability to different problem instances having different characteristics as well as different problem domains. With this goal in mind, machine learning techniques are vital for hyper-heuristics to make the right choices during the heuristic selection process. Learning can be achieved in an offline or online manner. An offline learning hyper-heuristic requires training over a set of problems, before it is used to solve the unseen problem instances. For example, Burke et al. (2006) use a case based reasoning system as a hyper-heuristic for solving course and examination timetabling problems. An online learning hyper-heuristic learns through the feedback obtained during the search process while solving a given problem. Most of the existing online learning hyper-heuristics incorporate reinforcement learning (Kaelbling et al., 1996; Sutton et al., 1998). A reinforcement learning system interacts with the environment and changes its state via a selected action in such a way as to increase some notion of long term reward. Hence, a learning hyper-heuristic maintains a utility value obtained through predetermined reward and punishment schemes for each low level heuristic. A heuristic is selected based on the utility values of the low level heuristics in Figure 1. A hyper-heuristic framework based on a single point search, where St denotes a candidate solution at time t, Hi is the i th low level heuristic, Rt is the resultant solution after applying a set of selected low level heuristics that goes into the move acceptance process